16 research outputs found
Interacting with Acoustic Simulation and Fabrication
Incorporating accurate physics-based simulation into interactive design tools
is challenging. However, adding the physics accurately becomes crucial to
several emerging technologies. For example, in virtual/augmented reality
(VR/AR) videos, the faithful reproduction of surrounding audios is required to
bring the immersion to the next level. Similarly, as personal fabrication is
made possible with accessible 3D printers, more intuitive tools that respect
the physical constraints can help artists to prototype designs. One main hurdle
is the sheer amount of computation complexity to accurately reproduce the
real-world phenomena through physics-based simulation. In my thesis research, I
develop interactive tools that implement efficient physics-based simulation
algorithms for automatic optimization and intuitive user interaction.Comment: ACM UIST 2017 Doctoral Symposiu
Recommended from our members
Efficient Acoustic Simulation for Immersive Media and Digital Fabrication
Sound is a crucial part of our life. Well-designed acoustic behaviors can lead to significant improvement in both physical and virtual interactions. In computer graphics, most existing methods focused primarily on improving the accuracy. It remained underexplored on how to develop efficient acoustic simulation algorithms for interactive practical applications.
The challenges arise from the dilemma between expensive accurate simulations and fast feedback demanded by intuitive user interaction: traditional physics-based acoustic simulations are computationally expensive; yet, for end users to benefit from the simulations, it is crucial to give prompt feedback during interactions.
In this thesis, I investigate how to develop efficient acoustic simulations for real-world applications such as immersive media and digital fabrication. To address the above-mentioned challenges, I leverage precomputation and optimization to significantly improve the speed while preserving the accuracy of complex acoustic phenomena. This work discusses three efforts along this research direction: First, to ease sound designer's workflow, we developed a fast keypoint-based precomputation algorithm to enable interactive acoustic transfer values in virtual sound simulations. Second, for realistic audio editing in 360° videos, we proposed an inverse material optimization based on fast sound simulation and a hybrid ambisonic audio synthesis that exploits the directional isotropy in spatial audios. Third, we devised a modular approach to efficiently simulate and optimize fabrication-ready acoustic filters, achieving orders of magnitudes speedup while maintaining the simulation accuracy. Through this series of projects, I demonstrate a wide range of applications made possible by efficient acoustic simulations
AirCode: Unobtrusive Physical Tags for Digital Fabrication
We present AirCode, a technique that allows the user to tag physically
fabricated objects with given information. An AirCode tag consists of a group
of carefully designed air pockets placed beneath the object surface. These air
pockets are easily produced during the fabrication process of the object,
without any additional material or postprocessing. Meanwhile, the air pockets
affect only the scattering light transport under the surface, and thus are hard
to notice to our naked eyes. But, by using a computational imaging method, the
tags become detectable. We present a tool that automates the design of air
pockets for the user to encode information. AirCode system also allows the user
to retrieve the information from captured images via a robust decoding
algorithm. We demonstrate our tagging technique with applications for metadata
embedding, robotic grasping, as well as conveying object affordances.Comment: ACM UIST 2017 Technical Paper
PodReels: Human-AI Co-Creation of Video Podcast Teasers
Video podcast teasers are short videos that can be shared on social media
platforms to capture interest in the full episodes of a video podcast. These
teasers enable long-form podcasters to reach new audiences and gain new
followers. However, creating a compelling teaser from an hour-long episode is
challenging. Selecting interesting clips requires significant mental effort;
editing the chosen clips into a cohesive, well-produced teaser is
time-consuming. To support the creation of video podcast teasers, we first
investigate what makes a good teaser. We combine insights from both audience
comments and creator interviews to determine a set of essential ingredients. We
also identify a common workflow shared by creators during the process. Based on
these findings, we introduce a human-AI co-creative tool called PodReels to
assist video podcasters in creating teasers. Our user study shows that PodReels
significantly reduces creators' mental demand and improves their efficiency in
producing video podcast teasers
ReelFramer: Co-creating News Reels on Social Media with Generative AI
Short videos on social media are a prime way many young people find and
consume content. News outlets would like to reach audiences through news reels,
but currently struggle to translate traditional journalistic formats into the
short, entertaining videos that match the style of the platform. There are many
ways to frame a reel-style narrative around a news story, and selecting one is
a challenge. Different news stories call for different framings, and require a
different trade-off between entertainment and information. We present a system
called ReelFramer that uses text and image generation to help journalists
explore multiple narrative framings for a story, then generate scripts,
character boards and storyboards they can edit and iterate on. A user study of
five graduate students in journalism-related fields found the system greatly
eased the burden of transforming a written story into a reel, and that
exploring framings to find the right one was a rewarding process